Video surveillance is of critical concern in many areas of life. One problem with video as a surveillance tool is that it may be manually intensive to monitor. One approach to increase the detection of various events involves the use of virtual boundary lines, i.e., virtual tripwires, in a video region of interest. However, prior art tripwire approaches are based on the assumption that the surveillance system is able to perform accurate object detection and tracking tasks. Based on the detected object trajectory in the visual input, those systems are able to determine if the tripwire is crossed by detecting an intersection between object trajectory and the virtual boundary. However, this prior art solution is limited because many virtual boundary crossings are not detected correctly in cases of: crowded scenes, objects that are very close together, or objects connected by long shadows, etc.